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ABSTRACT 

Optimal control of systems governed by delayed-dif ferential 
equations is explored by using the control theory developed for 
system.s governed by ordinary differential equations. A simple 
algorithm for producing a suboptimal control law with restricted 
feedback is presented. Two examples illustrate the computational 



method . 



FOREWORD 



This report summarizes a research effort which has been 
funded by the Foundation Research Program of the Naval Postgraduate 
School, Monterey, California. 
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I . INTRODUCTION 



A. DELAYED-DIFFERENTIAL EQUATIONS 

In many physical systems, such as rocket engines [1,2], 
antirolling stabilization systems for destroyers [3] and 
pipeline recycling systems for chemical reactors [4], the 
most accurate mathematical representation of system 
dynamics is found to be a matrix system of differential 
equations with finite time delay in the arguments of some of 
the state variables. 

x(t) = Ax(t) + Bx(t-x) + Cu(t) (1) 

t ^ 0 , T ^ 0 

x(t) = (fi(t) ; ^ (2) 

Equations of this form have been known as differential- 
difference, hysterodif ferential, functional differential, 
differential with transport lag, differential with deviating 
arguments, and delayed-dif ferential equations, to name just 
a few. For the purposes of this research, equations of the 
form of (1) are referred to as delayed-dif ferential 
equations . 

B. THE OPTIMAL REGULATOR PROBLEM 

The research to be described is concerned with a solution 
of the optimal regulator problem. Simply stated, this 
problem seeks the control function u*(t) (a member of the 
set of admissible controls u(t)) which minimizes a quadratic 
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cost function of the form 



J(<l>(t) ,x(t ) ,u(t) ) = / [x*(t)Qx(t) + u' (t)Ru(t ) ]dt > 0 (3) 

0 

subject to the constraints of equations (1) and (2). Q is a 

2 2 
n positive semidefinite matrix; R is a S. positive definite 

matrix; ^ n . 

C . NOTATION 

Notation follows that established by Bryson and Ho [5]- 
In general, it is as follows: 

1. Column vectors are denoted by lov/er case letters. 

Single subscripted lower case letters are elem.ents of 
vectors . 

2. Matrices are indicated by upper case letters. Doubly 
subscripted lower case letters represent elem.ents of 
a matrix. 

3. Matrix transpose is indicated by the prime symboK*). 

The follov/ing are deviations from the general rules 

established above. 

1. t represents the scalar Independent variable time. 

2. T denotes the scalar time delay, possibly a function 
of time and state variables. 

3. J is a scalar function of matrices. 

The lower case letters i,j,Jl,m, and n are used as 
indexing variables. 

5. The total derivative is indicated by d. 

6. A damped natural frequency is represented by w.. 

J 



D. PROBLEM STATEMENT 
1 . Introduction 

Inclusion of the delay term x in the state variable 
equation greatly complicates solution of the optimal 
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regulator problem. The well established theory of ordinary 
differential equation systems can not be applied with 
equation (1). In addition, although the necessary theory 
and computational procedures have been developed [6] for 
computation of the optimal control law, implementation of 
that law is often found to be either impractical or undesir- 
able. Implementation of the optimal control law requires 
feedback of a continuum of states, x(c), t-T£?<_t , 
which are often not readily available from the system being 
controlled. 

2 . Purpose 

The purpose of this research is to develop a method • 
for approximating the optimal control law for delayed- 
differential systems by utilizing the optimal control 
formulation for ordinary differential systems. In addition, 
a simple algorithm is offered for restricted feedback cases. 

E. BACKGROUND 

1. General 

Anderson and Soudack [7] indicate that the study of 
the delayed-dif ferential equation appears to have started 
with John Bernoulli in 1728, but most presently available 
information has been published since 19^0. Bibliographies 
published by R. Weiss [8] in 1959 and Chosky [9] in I960 
contain references to more than 350 separate papers dealing 
with the delayed-dif ferential equation and associated 
problems. The annotated bibliography by R. Weiss surveys 
English language papers published between 1935 and 1958. 
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The work by Chosky Is supplemental to the bibliography of 
Weiss and covers the period of 1916 to 1959, including 
reference to numerous foreign language papers. 

Prior to I960, several papers were published which 
addressed the problem of approximating the delay term so 
that the desired control law could be obtained. 

In a paper published in 19^0, Mason and Philbrick [10] 
devised a liquid level analog to a thermal process with time 
lag. This model was used to examine response to various 
types of control systems. A related work by Ziegler and 
Nichols [11] considered various control settings for auto- 
matic control circuits with process lag. The optimal control 
was determined from the area under a state time-history 
curve. Bretol [12] and Mlnorsky [3] used first order differ- 
ential equations to approximate control systems with time 
lags. After the control law was determined from the 
approximation, the actual system was adjusted to give the 
desired results. 

In an attempt to explain why, under certain circum- 
stances, a pendulum began to spontaneously oscillate with a 
frequency higher than its ovm damped natural frequency or 
that of the externally applied forcing function, Mlnorsky [13] 
expressed the system model as 

x^(t) + a^^x^Ct) + b^x^(t-x) + c^x^(t) = 0 (4) 

Considering the time delay term x to be small, x^(t-x) v;as 
expanded in a Taylor series to yield 



(-1)*^ ^ + ... - TX^(t) + X^(t) 

For very large n and small x this expression converges to a 
term of the form e Replacing the delayed term in equation 
(4) by e~^ the following system model was obtained; 

x^(t) = (a^ - b^e”*^) x^(t) + c^x^(t) = 0 

This equation was studied to determine regions of stability. 

The major work In the theory of delayed-dlf ferentlal 
equations has been undertaken since I960. This increasing 
emphasis may in part be attributed to the rising application 
of such equations to automatic control systems as well as 
the development of high speed computers. 

2. Development of Theory 

Bellman [l4] and El'sgol'ts [15] discussed the 
existence and uniqueness of solutions to equations of the 
form of ( 1) . 

Krosovskii [l6], appears to have been one of the 
earliest developers of an optimal control formulation for 
the linear quadratic problem with time delays in the states. 
Although he gave no explicit definitions for the parameters 
required to actually determine the optimal control law; the 
feedback was in the form of a linear combination of the 
states. Extension of Pontryagin’s maximum principle to 
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systems with multiple delays was accomplished by Kharatlshvll 
[17>18]. These works have served as a basis for numerous 
other papers on differential-difference equations. Orguz- 
toreli [ 19 ], Chyung and Lee [20] have also worked with the 
derivation of necessary conditions for optimality in the 
form of the maximum principle. 

Ross and Flugge-Lotz [6] developed an optimal linear 
control law, similar in form to Krosovskil' s , for a system 
in which the time delay had been normalized to one. (In a 
later work, Ross [21] showed that a system with any number 
of delay terms can be transformed into a system with one 
delay which can be normalized to one.) This paper also 
presented the sufficient conditions for the existence of 
admissible control functions and sufficient conditions for 
a linear control law to be optimal. Kushner and Barnea [22] 
showed that a linear control law for (1) subject to a qua- 
dratic cost function in the form of (3) was optimal with 
respect to the class of square Integrable control laws, if 
the solution to (1) was continuous. 

3 . Approximation of Delayed-Dif ferential Equations 

Because of the complexity of solving the optimal 
regulator problem when the system dynamics involve delayed- 
differential equations, considerable effort has been 
directed toward approximating the delayed-dif ferential 
equation by systems of ordinary differential equations. 

Repin [23] has shown that equation (1) can be 
approximated to any desired accuracy by a series of 
differential equations. 
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For systems in which the delay term is small, Jen- 
Wei [ 24 ] developed a procedure for replacing the delay term 
by a series of piece-wise linear functions which lead to 
a system of linear state variable equations with no time 
delay. Hess [ 25 ] has extended this procedure to a system 
in which the delay was not restricted. 

Results similar to those obtained by Ross and Flugge- 
Lotz [ 6 ] were developed by Soliman and Ray [ 26 , 27 ] by 
approximating the delay with an arbitrarily large set of 
differential equations. Similar procedures have been used 
by Westdal and Lehn [ 28 ] for a related problem. 
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II. THEORY OP THE OPTIMAL CONTROL OF 
DELAYED-DIFFEREMTIAL SYSTEMS 



A. EXISTENCE AND UNIQUENESS 

For an equation of the following form, 

x(t) = f(t,x(t) ,x(t “ t)) 
x(t) = 4>(t) , tQ-x 1 t _< tQ 
x(t,x(t)) 0 

El*sgol*ts [ 15 ] has shown that if the functions f, (j), x are 
continuous, the solution exists. If in addition 



0f 




9f 




d(J)(t ) 


, and 


dx 


9 (x(t ) ) 


9 


9(x(t - x) ) 


5 


dt 


dt 



are bounded in the vicinity of the initial value (a Llpschltz 
condition in x(t)), then the solution is also unique. 

Since it has been shown [6] that the optimal control, 
u*(t), is of the form 

u*(t) = -R“-^C’ [KQx(t) + / K, (e)x(t + 0 )d0 ] 

-X 

the solution to the optimal regulator problem (equations 1-3) 
exists and is unique if the above Llpschltz condition is 



satisfied . 



B. CONTROLLABILITY 



L. Weiss [ 29 ] developed algebraic sufficient conditions 
for delayed-differential equations with time varying coeffi- 
cients to be controllable. These conditions reduce to the 
ordinary algebraic criteria for controllability as the delay 
becomes negligible. The paper presents a method for computing 
the columns for the controllability matrix. 

Considering a system with constant coefficients. Hewer 
[ 30 ] has shown that if the system in which x-»-0 is control- 
lable, then the delayed equation is controllable for any 
T ^ 0. The converse of this statement is not always true. 

For the time invariant case, the results obtained by Weiss 
can be reduced to those obtained by Hewer. 

C. DEVELOPMENT OF THE OPTIMAL CONTROL LAW 

Ross and Flugge-Lotz [6] and Ross [21] have developed 
the optimal control law for equation (1) subject to (3) as 
follows ; 

1. Admissible Control 

If (1) is controllable, the set of admissible control 

. jnL^i 

functions, u(t), are those measure abd:e ' functions in L 2 [ 0 ,“] 
which yield a finite value for (3) for any set of initial 
conditions 4>(t). 

2 . Sufficient Conditions for a Linear Feedback 

Control Law to be Optimal 

The optimal control law of (1) subject to the cost 
function (3) and a specified set of initial condition 
functions (2) is of the form 
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(5) 



u«(t) = -R“^C [KQX(t ) + / K^(0)x(t + e)d0] 

-T 

t>^0 ; x(t) = 4>(t) , < 0 



if 



a. 

b. 



u*(t) is a stable control law (admissible in this case) 
Kq (a symmetric positive definite matrix), K^(0) and 
K^(Q,Q) satisfy the following relationships 



A’Kq + KqA - KqCR ^C'Kq + K^»(0) + K^(0) + Q = 0 (6) 



dK, (0) 

^ = (A’ - KqCR“-^C’ )%(0) + K2(O,0) 



-T < 0 < 0 (7) 



0K2(c, 9) aK2(c,e) 



00 



9C 



= -K^' (C)CR"^C'K^(0) 



( 8 ) 



-T<C<0, -T<0<O 



K^(-t) = KqB 



(9) 



K2(-t,0) = B»K^(0) 



( 10 ) 



If B = 0, equivalent to a system with no delay term, the 
above conditions reduce to the familiar algebraic Riccati 
equation. The first term in u*(t) is the optimal control 
law for the undelayed system; the second and more complicated 
term accounts for the addition of the delay term. 
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Under the above conditions, the quadratic cost function 
(3) can be expressed In terms of the Initial condition 
function (2). 

0 

J((J),u*) = <j>’ (0 )Kq 4>(0) + 24>»(0) / K^(0)({.(e)d0 

-T 

0 0 

+ / / (C)Kp(C, 9)4>(0)d?d0 (11) 

-T -T 

Although the control law, u*(t), is optimal for an arbitrary 

\ 

initial condition function which is continuously differenti- 
able, the value of (3) is, as would be expected, a direct 
function of the initial conditions. 

3. Stability of the Optimal Control Law 

In order to apply the optimal control law, some 
approximate method of computation is required. Once the 
control law is calculated, there is no known algorithmic 
method by which the stability of the control can be 
assessed [6]. Determination of stability requires that the 
closed loop system performance be investigated for a set of 
initial condition functions large enough to cover those 
expected to be encountered by the control system. If no 
unstable behavior is detected, the control law can be 
assumed to be stable in that particular application. 
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III. APPROXIMATION OF THE OPTIMAL CONTROL LAW 
FOR DELAYED-DIFFERENTIAL SYSTEMS 



A. APPROXIMATION OP DELAYED-DIFFERENTIAL EQUATIONS 

Repin [ 23 ] has shown that equation (1) can be approxi- 
mated by a series of equations, determined by defining a new 
set of state variables. The state variables are defined as 
follows : 



x^^(t) = x(t) 
x^Ht) - x(t - 



X.*(t) = x(t - 
1 m-1 



( 12 ) 



Xm*(t) = x(t - t) 

Where m Is a positive Integer equal to or greater than two, 
x^*(t) Is a nxl column vector composed of the elements of 
the original state vector. 

Using simple forward difference formulas for estimating 
derivatives, the following set of equations can be developed: 
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Xj^*(t) =Ax^*(t) +BXj^*(t) +Cu(t) 



k^Ht) = (x^*(t) - X 2 *(t)) ^ 



Xj_*(t) =(Xj^_*(t) - X^*(t)) 



( 13 ) 



m-1 



V(t)' - V(‘)> ^ 



This series of first order, linear, matrix differential 
equations can be shown to be expressible as 



y(t ) = Aq y(t ) + Cq u(t ) 



(IM 



where 



(m-1) j -(m-1) j 

T T 



(m-1) 



0 

0 

(m-1) 



B 



I . . . 0 



0* ... I 



(15) 





1 


0 • • • 


o 




'c' 


Mi 

O 


0 


0 • • • 

• 

• 


o 


o 

o 

Ml 


0 

• 

• 




r 

O 


0 • • • 


o' 




1 



( 16 ) 
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y(t) 



( 17 ) 





Repin has shown that by selecting m large enough, 
equation (1) can be approximated to any degree of accuracy 
desired, and in the limit as m -*■ « , the representation 
presented above becomes exact. 

B. OPTIMAL AND SUBOPTIMAL CONTROL OF 
ORDINARY DIFFERENTIAL SYSTEMS 

For the time invariant linear system of differential 
equations 

x(t) = Aq x(t) + Cq u(t) j t ^ 0 ; x(0) = (p 

with a quadratic cost function in the form of (3), the opti- 
mal control law has been shovin to be 



where V is the symmetric positive definite solution to the 
algebraic Riccati equation 



u*(t) = -R"^ Cq’ V x(t) 



( 18 ) 




( 19 ) 
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When the optimal control law is applied, the value of 
the cost function (3) can be expressed as 

J = (J)’ V4) 

Elkind and Falb [31] have shown that if some control 
u(t) = -Gx(t) (G is a matrix of constants conformal to x(t)) 
is applied, instead of the optimal control law (18), the cost 
function can be expressed as 

Jq = 4> ’ Vq 4 . > 4) • V <f. (20) 

further, if the eigenvalues of (Aq - Cq G) have negative real 
parts, Vq is the symmetric positive definite solution to 

(Aq - Cq G)'Vq + Vq(Aq - Cq G) + G’ G + Qq = 0 (21) 

These expressions can be used to determine the value of 
(3) for any stable feedback composed of a linear combination 
of states. 

C. APPROXIMATION OP THE CONTROL LAW 

It has been shown that (1) can be approximated by a 
system of differential equations 

Ht) = A^y(t)+ CQu(t) ; t ^ 0 (22) 

y(t) = ? ; t = 0 

with y(t), y(t), Aq, and Cq as defined by (12-17). The 
initial condition matrix is 
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( 23 ) 



4>(0) 



oo 

/ [y'(t)Q«y(t) + u'(t)Ru(t)] dt 
0 ^ 

(2H) 

The hypothesis here is that an accurate approximation to 
the optimal control law (5) for the delayed-dif ferent ial 
system (1) can be obtained from the optimal control lav; (l8) 
for an ordinary differential system. This ordinary differen- 
tial system is, of course (22), the one which approximates 
the delayed-dif ferential system (1). 

Rigorous proof of this hypothesis is difficult. Instead, 
the following section shows that this approximate method is 
exactly equivalent to the computational approximation which 
Ross utilizes in solving for his optimal control law for 
system (1). 

D. EQUIVALENCE OP COMPUTATIONAL APPROACHES* 

1 . Ross’s Method 

In order to compute the optimal control law (5)> 

Ross [21] replaces the conditional equations (6-10) by 

*For convenience, and without loss of generality, the 
time delay t has been normalized to unity in the discussion 
which follows. 



The cost function is 

J(y(0),y(t) ,u(t)) = 
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finite difference formulas on the surface -1 £ 0 < 0 , 
-1 < C < 0 . The resulting equations are 



A’K 



Q + Kq A- Kq CR"^ C Kq + (0) + K^^CO) + Q = 



( 25 ) 



K(n^) - >'i(-si-)} (iS^l ' {*' - Kq O' } K, 






m-1 



+ K, 0 , 



-(i-1) 



m-1 



1 < i < m-1 



( 26 ) 



K( 



-d-i) -(.1-1) 



2 m-1 



m-1 



- K, 



-1 -(J-1) 

m-1 * m-1 



+ K, 



^ -(i-1) 
^ m-1 



--( j- u] _ K hi i-ji /mi 

m-1 J ^2 [ m-1 * J 



= -K ' CR"^C'K, 

1 [ m-1 J 1 



f-(.1-l)l 



m-1 



1 £ i _< m-1 
1 < j < m-1 



(27) 



K^(-l) = KqB 



^2 ("^ » 



= B' K, 



jzl. 

m-1 



( 28 ) 



, 0 < j < m-1 (29) 



with A, B, C, Q, and R the same matrices appearing in (1) and 

(3). 
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Now for the same value of the Integer m, let K be the 
positive semldeflnlte solution of the algebraic Rlccatl 
equation 



A ’ , K + K , - K C C' T K + T =0 (30) 

m— 1 m— 1 m— 1 m— l m-'l 



where 



m-1 



A 0 0 

(m-l)I -(m-l)I 0 

0 (m-l)I -(m-l)I 



• • • 



• • • 3 

... 0 

... 0 

(m-l)I -(m-l)I 





Q 


0 


o" 




c 


Q -I ” 
m-1 


0 


0 

• • • 


0 


C T = 
m-1 


0 

• 

• 

• 




0 


0 . . 


. 0_ 




p _ 



(31) 



(I Is the n X n Identity matrix, A^_^ and are square 

matrices having n x m rows and columns, and Is n x m by £) 

If one considers the (symmetric) matrix K to consist of 
m^ sub-blocks, , for 0 _< 1 m-1, and 0 £ j m-1 , of 

n X n _ 





o 

o 


•'o.l 


• • • 


1 — 1 

1 

S 

o 


K = 


o 

*> 

1 — 1 


’'1,1 

• 


• • • 

• • 


K-, T 

l,m-l 




IVi 


• 


• • 


^m-1 ,m-l 



(32) 



then (30) becomes a set of algebraic equations In the unknowns 



K 



ij* 
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Ross then points out that if one dissects (30) and 
compares those algebraic relations with the algebraic rela- 
tions (25-29), the two sets of equations are found to be in 
direct correspondence. Specifically: 



^0 ~ ^ 0,0 






(m-1) K 



0,i+l » 



Kl(-l) = KqB = 



(33) 



0 < i < m-2 



(34) 



K, 



-1 -j 

(m-1) ’ ( m-1 ) 



- (m-1) , 0£i£m-2 and 

0 IJ 1 m-2 



K, 



K, 



-1 , 






(m-1) 



= B 






-1 



[WT) » 



-1 






0 1 j lm-1 



Olilm-1 (35) 



Thus, solution of the Riccati equation (30) is equivalent 
to solution of equations (25-29). Since efficient algorithms 
exist for the solution of Riccati equations of this form, 

Ross bases his computational approach on the solution of 
equation (30). Interpolation between the resulting values 



of K, 





r-ii 


for 0£l£m-l and K 2 




’ 1 


[m-lj 




m-1 ’ m-1 



for 



l,j = 0,1,,.., m-1 , yields approximations to Kq, K^(6) and 
K2(C,9) in equations (5) and (11). The computation is 
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terminated when no significant improvement in the approxi- 
mation of Kq, K^(0), occurs as m is Increased. 

2 . The Method of Section III. C. 

It has been shown that the optimal control law for 
(22) with a cost function of the form (24) is 

u«(t) = -R"^ Cq' Vy(t) • (36) 



where V is the solution of the Rlccatl equation 



Aq’ V + VAq - VCqR"^ Cq' V + Qq = 0 (37) 

2 2 

V is a symmetric (n m) matrix composed of m sub-blocks 

V 

±y 





V = 




V 

mm 



(38) 



The value of the cost function J is given by 



J = (|)' V (j) 



(39) 



Equations (36) and (39) can be rewritten 



U' 



(t) = -R ^Cq' [V^^y^(t) + ^I^V^iYiCt)] 



m 



(40) 
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m 



J = 4)’ (0) V,, 4>(0) + 2 4>' (0) y . 4) 
XX 1=2 



^1-1' 



m-1 



m m 

^ i 

J =2 1=2 



"l-l" 



4> 



[m-lj 'Ij 






m-1 



(^1) 



or 



m-1 



u*(t) = -R Cq’ Cv^^y^(t) + \,1+1*^"^ ’^1+1^^^ * m-1 ^ 

(i<2) 



r '1 T 



m-1 m-1 






" }h Ji ''i-i.j-i-'”-"' -ni^J 



(m-1)' 



(i<3) 



Now the Integrals In the optimal control law and cost 
function of equations (5) and (11) can be approximated by 
finite summations as follows: 



m-1 

u*(t) - -R"-^ C [K. x(t) + I 

^ 1=1 



m-1 

J 4>’ (0) K. 4>(0) + 24)' (0) I 
^ 1=1 
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Recalling that equations (12) and (17) state that 



y^(t) 




N 

1-1 

m-1 



1 1 ^ 2 ^ • • • y ill 



then each term in equations (^12-^3) corresponds to a term In 
(I|I}_ 1 | 5 ). This correspondence strongly suggests 



V 

'^ 1 , 1+1 



K. 



'zllzll 

. m-1 



1 

m-1 



V 

1+1, j+1 



K, 



^ -( 1 - 1 ) 
^ m-1 



m-1 

J 



(m-l)^ 



(il6) 



Assuming the validity of equations (^6), one can now approxi- 
mate the optimal control law for delayed-differential 
equations by utilizing the optimal control formulation for 
ordinary differential systems. 

3. Equivalence 

The validity of equations (46) and their equivalence 
to Ross’s solution can be proven by noting that the Rlccatl 
equation (30) which Ross utilizes to generate his solution is 
identical to the Riccatl equation (37) used to generate the 
differential solution. 

E. ADVANTAGES OF THE DIFFERENTIAL APPROACH 

There are certain advantages to be gained by using the 
differential approximation scheme In calculating the optimal 
control law for delayed-differential systems. 

First, the closed loop eigenvalues of the approximating 
differential system yield information about the quality of 
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the differential approximation, and about the stability 
characteristics of the actual, optimally controlled delayed- 
differential system. As Hess [25] points out, the largest 
closed loop damped natural frequency of the differential 
system should be small when compared to the frequency of 
occurrence of the station points used in defining X 2 *(t) 
through in (13) • This means ensuring that 



( 1 ) 



2TT(m-l) 



<< 1.0 



j=l,2, . . . ,nxm 



(47) 



■f" 

where oJj is the magnitude of the imaginary part of the j 
closed loop eigenvalue. The smaller the left hand side of 
equation (47), the more accurate are the forward difference 
expressions for X 2 *(t) through x^j^^(t) in (13)* If the 
restrictions of equation (47) are met, the closed loop 
eigenvalues of the differential system give excellent 
information regarding the stability characteristics of the 
actual, optimally controlled delayed-differential system. 

Second, the differential approach is amenable to systems 
with time varying parameters. This means that one should be 
able to find an approximation to the optimal control law for 
a delayed-differential system of the form 

x(t) = A(t)x(t)+ B(t)x[t - x(t)] + C(t)u(t) 
subject to a cost function of the form 
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J((J)(t),x(t),u(t)) 



-f 



[x’ (t) Q(t ) x(t) + u' (t ) R(t ) u(t ) ]dt 



by utilizing the theory of optimal control of ordinary time 
varying differential systems. 



F. ALGORITHM FOR RESTRICTED FEEDBACK 

1. Introduction 

The optimal control law (5) may require a digital 
computer for implementation. The question naturally arises 
as to just how well the system' ( 1 ) would perform with a 
simpler, suboptimal control law. For example, for systems 
in which the time delay is relatively small, a control law 
of the form 

u(t) = -G^ x(t) (^ 8 ) 

may yield satisfactory performance, and be much easier to 
implement than control law (5)« If law (48) is not 
satisfactory 

u(t) = -G^ x(t) - G 2 x(t-l) (49) 

or 

u(t) = -G^ x(t) - G 2 x(t-is) - G^ x(t-l) ( 50 ) 

might be considered. The next section offers a direct 
constructive procedure for determining the G^^ in (48-50) 
based upon the optimal formulation ( 5 )* 

2 . Restricted Feedback 

The method to be described is based upon the same 
linear approximations for the states of the delayed-dlf ferential 
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system as used in the differential approximation (13). The 
procedure involves representing x(t+6) in (5) by any number 
of straight line segments as follows: 



Zeroth approximation 
x(t + e) 5? x(t) , • 0 - r- 

First approximation 

x(t + 6) ?=■ x(t) + [x(t) - x(t-l)]6 , O£0_<-1 

Second approximation 

x(t+e) x(t) + ~ ^ -A. ^ ^ ^ ^ 

x(t+e) 2 s x(t-i 5 ) + [ 2 LLtzkl-zJLLtzll]( 6 +i 5 ) ^ 

Third approximation 

x(t+e) a? x(t) + -^ . LK-y ± l ]e , ± Q O 

x(t+ 6 )s 5 xCt-Vs) + ^3 ) -^x(t-V a. ) .3 ( 

x(t+e) x(t-2/3) + [ X ( ^-^ 4 ( g+2/3 ) ^ ~ 
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A simplified control law based upon any one of these 
approximations can be computed by substituting one of the 
above expressions for x(t+e) into the Integral expression 
in the optimal control law (5). For example, using the 
second approximation 



u*(t) -R”^ C [Kq x(t) + J K^(e){x(t) + 2[x(t) - x(t-}s) ]e} d0 



"h 



-h 






+2[x(t-%) - x(t-i) Ke+jj) }de] 



which yields 



-1 r ° 

u*(t) ^ -R C» i [Kq + / (1 + 20) K^(0) de] x(t) 

I -I5 



0 -h 

+ [- / 20 K, (0)d0+ / (2 + 20) K, (0) d0] x(t-5s) 

-k -1 ^ 

-h \ 

+ [- / (1 +20) K^(0) d0 ] x(t-l)| 



-G^x(t)- G 2 x(t-%) - G^x(t-l) 



Since Kq and K^(0) have already been calculated, the expres- 
sions above can be numerically integrated to yield a 
simplified, suboptimal control law. 

The performance of the delayed-dif ferentlal system 
with any of these suboptimal control laws depends, of course, 
upon how well the straight line approximations represent 
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x(t+ 6 ). In this light, if a suboptimal law based upon the 
zeroth approximation yields a significantly larger value of 
the cost function than the optimal law, a higher order 
approximation (rather than a different value of is 

needed. The large increment in the cost function is 
indicating that 

x(t + 0 ) x(t) ; 

is not a satisfactory approximation. 

3. Stability and Cost Function Evaluation 

Information about the stability of the suboptimal 
control law can be obtained from the closed-loop eigenvalues 
of the differential system as outlined in Section III. E. 

The cost function can be evaluated either directly from 
simulation or in approximate fashion by using expressions 
similar to ( 20 - 21 ). 
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IV. SAMPLE CONTROLLER DEVELOPMENT 



A. SYSTEMS TO BE STUDIED 
The scalar equations 

x(t) = a^^x(t)+ b^^x(t-l)+c^^ u(t ) (51) 

x(t) = 1+t , (52) 

00 

J = / (x^(t) + u^(t))dt (53) 

0 

were studied for a^^ = = -^ 2.1 ~ (Case 1) and 

' ♦ 

a^^ = b^^ = -Cq_-j_ = -2 (Case 2). 

For Case 1, the required order of the differential 
approximation was determined to be ten; for Case 2, the order 
was determined to be fourteen. These values for m Insured 
that to./(2TT(m-l) ) 2? .1 , J = l,2,...,m 

J 

The differential approximations, expressed In matrix 
notation are: 



y(t) 


~ J 




Cq u(t) 


, t ^0 




( 5 ^) 


y(o) 


= 1 










(55) 


J 




00 

/ (y’ (t) Qpi y (t ) 
0 ^ 


+ u^(t))dt 




(56) 
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. . . 0 


1 — 1 
f—i 








(m-l) • 


-(m-l) 


. . . 0 


0 
















(57) 


u 
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(m-l) 


. . . 0 


0 
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• 

• 

0 


(m-l) 


-(m-l) 





3^1 



Cq' = 0 ... 00] 

?» = [1 (m-2)/(m-l) . . . (m-l-l)/(m-l) . . . 0] , 

1 0 ... o' 

0 0 ... 0 

% = 

0^0 0 



B. FORMULATION OP THE OPTIML CONTROL LAW 

The optimal control law for the delayed-differential 
system (51-53) was obtained by utilizing the differential 
approximation (5*^-56). The matrix Riccati equation (19) 
for the differential approximation was solved and the resulting 
used to determine Kq and K^(0) via 



K 



1 



-1 ' 
m-1 

J 



(m-1) V 



1,1+2 



i=0,l,2, . . . , (m-2) 



K^(-l), not normally available from the differential approach, 
was obtained from Ross's solution (equation (3^)) as 

K^(-l) = 

The resulting Kq, K^(0) and cost function values are shown 
in Figures 1 and 2 for Cases 1 and 2 respectively. Figure 3 
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shows x(t) for the uncontrolled and optimally controlled 
delayed-differential system for Case 1. Figure 4 presents 
the optimal control function, u*(t), for Case 1. Figures 
5 and 6 show similar results for Case 2. 

C. FORMULATION OF A RESTRICTED FEEDBACK CONTROL LAW 

Suboptlmal control laws were developed for feedback of 
a restricted set of state variables associated with the 
differential approximation (51-57) by the method presented 
in III. F.2. These control laws are: 

I 

Zeroth approximation 
Case 1 

u(t ) = 2l6 x(t ) 

Case 2 

u(t) = 196 x(t ) 

First approximation 
Case 1 

u(t) = -.3^0x(t)+ .123x(t-l) 

Case 2 

u(t) = -.^09x(t)+ .213x(t-l) 

Second approximation 
Case 1 

u(t) = -.38ilx(t)+ .0887 x(t-} 2 ) + .079 x(t-l) 

Case 2 

u(t) = -.4il78x(t)+ .115x(t-J^+ .155x(t-l) 
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Third approximation 



Case 1 

u(t) = -.392 x(t)+ . 0384 x(t-. 33) + .0803 x(t-. 6?) 

+ .0570 x(t-l) 

Case 2 

u(t) = -.445 x(t)+ .024 x(t-.33) + .128 x(t-.67) 

.+ .115 x(t-l) 

The value of the cost function vs. the number of states 
In the restricted feedback Is presented In Figures 7 and 8 
for Cases 1 and 2, respectively. Figure 9 presents x(t) for 
the delayed-dlf f erentlal system of Case 1 with no control 
and with the control law determined from the third approxi- 
mation above. The control function for the third approxima- 
tion Is shown In Figure 10. Figures 11 and 12 present 
similar results for Case 2. In both cases, the use of a 
fourth state In the feedback (third approximation) produced 
less than a one percent Improvement In the value of the cost 
function. 

In order to compare the restricted feedback gains 
obtained by the algorithm of Section III.F.2 with optimal 
gains associated with feedback composed of one and two 
states, a gradient optimization technique was mechanized. 

In this technique, equations (20) and (21) were solved 
recursively to yield the optimum values of the restricted 
feedback gains for the differential systems of Cases 1 and 2. 



A more efficient method for accomplishing this optl 
mlzatlon can be found In Ref. 32. 
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A check was then made to ensure that these gains were the 
optimal restricted feedback gains for the actual delayed- 
dlfferentlal system by direct integration of equations (51) 
and (53). The results are 

Case 1 

u(t) = -.300 x(t) 

J = . 392110 

Case 2 

u(t ) = -. 500 x(t ) 

J = .3729 

Case 1 

u(t) = -.330 x(t)+ .130x(t-l) 

J = .37764 

Case 2 

u(t) = -.400 x(t)+ .225 x(t-l) 

J = .3314 

Comiparison of these results v;ith those obtained from the 
zeroth and first order approximations indicate that the 
algorithm v;as moving the restricted feedback gains in the 
right direction as the order of the approximations was 
increased. The utility of the algorithm is also borne out 
by Figures 7 and 8. 
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V. CONCLUSIONS 



This research has shown that the optimal control law 
for a delayed-dif ferential system can be determined to any 
desired accuracy from the optimal control law for an ordi- 
nary differential system. Since the theory associated 
with differential systems is well established, and efficient 
algorithms for the required computations are readily avail- 
able, this approach has obvious merit. 

Further, it has been demonstrated that a simple algorithm 
can be used to determine a suboptimal, restricted feedback 
control law. 

The closed loop eigenvalues of the differential approxi- 
mation can be used to yield information about the conver- 
gence of the approximation. Although these eigenvalues 
completely determine the stability of the differential 
approximation, they can only suggest, albeit strongly, that 
the resulting control law will produce a stable delayed- 
differential system. Further research is required to 
ascertain the conditions under which a control law determined 
by this method will definitely produce a stable delayed- 
differential system. 
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FIGURE 1 

K^(6) vs 0, Case 1 
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FIGURE 2 

K^(0) vs 0, Case 2 
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FIGURE 3 



System Response With No Control and With Optimal Control, 

Case 1 
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FIGURE 4 

Optimal Control Function, Case 1 
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FIGURE 5 

System Response With No Control and V/lth Optimal Control, 

Case 2 



U*(t) 




FIGURE 6 

Optimal Control Function, Case 2 
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Value of Cost Function 




Number of States in Feedback 
FIGURE 7 

Value of Cost Function vs. Number of States in Restricted 

Feedback, Case 1 
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Value of Cost Function 




FIGURE 8 

Value of Cost Function vs. .dumber of States in Restricted 

Feedback, Case 2 
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System Response With No Control and VJith Restricted Feedback 

Case 1 
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FIGURE 10 

Restricted Feedback Control Function (Third Approximation) 

Case 1 



49 



x(t ) 




System Response With No Control and With Restricted 

Feedback Control, Case 2 
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FIGURE 12 

Restricted Feedback Control Function (Third Approximation) 

Case 2 
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